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Abstract 

Through a straightforward Bayesian approach we show that un¬ 
der some general conditions a maximum running time, namely the 
number of discrete steps performed by a computer program during its 
execution, can be defined such that the probability that such a pro¬ 
gram will halt after that time is smaller than any arbitrary fixed value. 
Consistency with known results and consequences are also discussed. 


1 Introductory remarks 

As it has been proved by Turing in 1936 P], if we have a program p running 
on an Universal Turing Machine (UTM), then we have no general, hnite and 
deterministic algorithm which allows us to know whether and when it will 
halt (this is the well known halting problem). That is to say that the halting 
behavior of a program, with the trivial exception of the simplest ones, is not 
computable and predictable by a unique, general procedure. 

In this paper we show that, for what concerns the probability of its halt, 
every program running on an UTM is characterized by a peculiar asymptotic 
behavior in time. Similar results have been obtained by Calude et al. |2] 
and by Adamyan et al. jHI through a different approach, which makes use of 
quantum computation. 

*First version: March 2006. Article in press. Chaos, Solitons & Fractals (2006), 
doi: 10.1016/j.chaos.2006.08.022. 
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2 Probabilistic approach 

Given a program p of n bits, it is always possible to slightly change its code 
(increasing its size by a small, hxed amount of bits, let’s say of s bits with 
s n) in order to include a progressive, integer counter that starts counting 
soon after p starts to run on an UTM and stops, printing the total number 
of steps done by the UTM during the execution of p, soon after p halts. Let 
us call this new program p'. Its algorithmic size is then of n + s bits. 

As it has been said in the previous Section, for an arbitrary program of 
n bits no general, hnite and deterministic procedure exists that allows us to 
know whether such program will ever halt or will keep running forever on 
an UTM Thus, in our case, we have no hnite, deterministic algorithmic 
procedure to decide whether and when p' will print the integer number of 
steps done by the UTM during the execution of p, until the halt. 

Let us now make a brief digression on algorithmic complexity. Suppose we 
have a (randomly chosen) binary string of n bits. Its algorithmic complexity 
(for a rigorous dehnition of algorithmic complexity, see |3]) is less than or 
equal to n+c bits, where c is a specihed constant in the chosen language. The 
case of a self-delimiting UTM (see jl]), for which the algorithmic complexity 
of a program of n bits is less than or equal to n + 0 {\og 2 n) bits, is dealt with 
later. The a priori probability Pi that its algorithmic complexity is equal to 
k bits^, with fc < n -1- c, is: 


2^ 

2n+c+l _ 2 ’ 

since the total number of possible generating strings less than or equal to 
n + c bits in size is — 2, while there are only 2^ strings of k bits. 

Now we want to calculate the somewhat different a priori probability P 2 
that a (randomly chosen) program of algorithmic complexity of k bits that 
produces an output^ generates just a string of n bits, with n + c>k. 

From Bayes theorem we have that such probability can be calculated as 
the ratio between Pi of eq. m and the sum of all the probability Pi, with 
n + c ranging from k to inhnity: 


P 2 


2n+c+l_2 
2k 


E oo 

i=k 2»+l—2 


( 2 ) 


^And thus, following the definition of algorithmic complexity the a priori probability 
that a minimum program of k bits exists that generates the string of n bits as output. 
^Note that this condition is crucial. 
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The probability P 2 can be easily rewritten and simplified as follows: 


P 2 = 


1 


^n+c+l 




1 


i=k 2*+l-2 


‘2n+c+l 


‘2n+c—k+l 


n + c>k, (3) 


since the snm 'Yj^k 2 »+i _2 safely approximable to 1/2^ for k > 10. 

Snppose now that we have a program p' of algorithmic complexity of k bits 
that will halt for snre, even if we are not able to know if and when it will do 
so by simply inspecting its code, and that contains a discrete step connter 
which connts the total nnmber of steps done by the program nntil the halt 
and prints it as ontpnt^. Applying Bayes theorem again, the probability that 
the ontpnt string is of a size greater than or eqnal to m bits (with m + c > k) 
is: 


P(size > m) = 


E OO 
S — 


2k 


s=m 2®+^+i—2 

E CO 2 ^ ’ 

i=k 2^+^—2 

Using the same approximation done in eq. 


comes: 


m + c> k. (4) 

dSl) the above eqnation be- 


P(size > m) ~ 


1 

Q^m+c—k ’ 


m + c> k. 


(5) 


Thns, the probability that the ontpnt string is less than m bits in size is: 


P(size < m) = 1 — 


E OO 
S— 


00 2 ^ 

s=m 2^+c+1-2 


E oo 

i= 


2k 


~ 1 — 


i=k 2»+l-2 


2^m+c 


-k ' 


m + c> k. 


( 6 ) 


It is easy to verify throngh a nnmerical check that for m + c > fc + 50, 
and thns even better for m>k + 50, the valne of P(size < m) is practically 
eqnal to 1. 

This means that if a. program p of algorithmic complexity of k bits halts, 
then it will do so in snch a way that the total nnmber of steps is a nnmber 
of size less than or eqnal to A: + 50 bits with very high probability. Thns, the 
decimal nnmber of steps t done by the program before the halt (the discrete 
time before the halt, or the characteristic time) can not be more than 
with very high probability. 

The above analysis and results hold also if the program p has a size of 
k bits, but a smaller algorithmic complexity. In such cases the characteristic 
time 2^^^^ provides a more conservative halting time. 

^For every size k we can add few instructions to implement the counter, altering only 
slightly its length, as explained at the beginning of this Section. 
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If we consider a self-delimiting UTM, then the algorithmic complexity of 
a program of n bits is less than or eqnal to n -|- 0(log2 n) bits (see jl]). With 
snch complexity measnre eq. o becomes 


nk 

p _ _ 

^ 2n+0{log2n)+l _ 2’ 

and eq. 0 conseqnently becomes 


P 2 


_ ^ _ 

2 n+ 0 (log 2 n) + l _2 

E oo 2 ^ 

i=l 2 *+C>{log 2 *)+l _2 


where I is an integer snch that I + 0 (log 2 1) = k. 
Eqnation © then becomes 


(7) 

( 8 ) 


P(size < m) = 1 - 2 -+c;aog 2 .)+ ^^ m + Oi\og,m) > k. (9) 

A^i = l 2^+0{log2 + l — 2 

If we consider cases with I S> 0(Iog2/) and m 3> 0(log2m), then eq.® 
can be safely approximable by 


P(size < m) ~ 1 


E CO 

s=r 


2s+i_2 


E CO 

i=. 


2k 


~ 1 


k 2 ^+ 1-2 


2^m—k ’ 


m > k. 


( 10 ) 


Thus, for I 3> 0(log2/), where I + 0(log2/) = k, and m 3> 0(log2m), 
P(size < m) ~ 1 — 2^“™ and again the value of P(size < m) is practically 
equal to 1 for m > fc -|- 50, as before. 

Actually, it is possible to prove^ that for a suitable constant b > 1 the 
function 1 — is a lower bound for the value of P(size < m) for both 

complexity measures, and thus for m > fc -|- 50 -|- 6, the value of P(size < m) 
becomes closer and closer to 1. 


Given eq. or eq. it is always possible to find a constant 6 > 1 such that: 


E oo 2 ^ 

s—m 2®+^—2 

E oo 2 k 

i—k 2bi+i_2 


1 - 


1 

^m—b—k 


< P(size < to) < 1. 


For m — b > k + 50, and thus for to > fc -t 50 -I- 6, the value of P(size < to) is practically 
equal to 1. 
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3 Apparent paradoxes and their solution 

Now, consider the following pecnliar case, which obvionsly provides a limita¬ 
tion in the application of eq. as every pecnliar case does with probabilistic 
approaches. 

It is possible to write a simple program p of [log 2 n \ + c bits that, given 
the decimal nnmber n, recnrsively calcnlates the nnmber 2”+^ — 2, connts 
from 1 to — 2 and then stops. As it is obvions, the program p rnns for a 
total nnmber of steps greater than 2"+^ — 2 and then stops, bnt eq. (jHl) rnay 
give an almost zero probability for snch a resnlt, since for snitable valnes of n 
the characteristic time 2l^^°®2"l+c+5i mnch mnch smaller than —2. 

As it always happens with probabilistic treatments, if, analyzing the code, 
we are able to know in advance that the program will halt and we are also 
able to know when it will halt, then eq. is of poor nse. Bnt this does 
not dismiss onr resnlts as meaningless: it is obvions that when we have a 
deterministic solntion of a problem, the probabilistic one simply does not 
apply. 

Another apparent paradox with the above resnlts is the following. It 
wonid seem possible to write a relatively small program h, again of [log 2 n] -|- 
c bits and with n 3> [log 2 n] -|- c, which lists all the 2”+^ — 2 programs of 
size less than or eqnal to n bits, rnns each program for a discrete time eqnal 
to 2^+®^, where k is the size of the program, and stores the ontpnt strings 
of the halting ones. Then, simply printing as ontpnt a string greater by one 
nnit than the greatest among those stored, the program h is able to provide 
a string of algorithmic complexity greater than or eqnal to n bits with a very 
high probability. This seems to challenge our results since n S> [log 2 n] -|- c 
and h would be able to print a string of algorithmic complexity greater than 
its own size. But it does not. 

As a matter of fact, among all the program of size less than or equal to 
n bits executed by h there is the program h itself, which we will call ^ 2 , 
and this fact generates a contradiction that does not allow the program h to 
produce any meaningful output. 

In fact, the size of ^2 is obviously equal to [log 2 n] + c bits and we 
already know that after a characteristic time of 2r*°S2A+c+5i g^gpg pg 

still running (the running time of h, and thus of ^ 2 , is obviously greater than 
2n+5i g^gpg^ should halt by definition and thus we surely have among 

the programs of size less than or equal to n selected by h as non halting, an 
halting one, making the procedure h meaningless. 

Besides, note that if we accept the output s of h as true, then the output 
of h has to be at least s -f 1, since also the output of /i 2 is equal to s, and so 
on, endlessly. 
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Stated in other words, it is not possible to use our results to write a me¬ 
chanical procedure able to print a string more complex than the mechanical 
procedure itself, as it should be according to the dehnition of algorithmic 
complexity. 

4 Mathematical implications 

Being able to solve the halting problem has unimaginable mathematical 
consequences since many unanswered mathematical problems, such as the 
Goldbach’s conjecture, the extended Riemann hypothesis and others, can be 
solved if one is able to show whether the program written to hnd a single 
hnite counterexample will ever halt UU- 

However, the estimate of the characteristic time done in the previous 
Section, namely t ~ where n is the size of the program, shows that the 

result obtained in eq. © is not much useful for the practical resolution of the 
halting problem, even for a probabilistic one, since almost all the interesting 
programs have a size much greater than 50 bits, giving astronomically huge 
characteristic times. 

Anyway, our result should be of some theoretical interest since it shows 
an asymptotic behavior typical of every Turing Machine. All this seems to 
shed new light on the intrinsic signihcance of the algorithmic size or, more 
precisely, of the algorithmic complexity of a program encoding a mathe¬ 
matical problem. As a matter of fact, such a low-level and low-informative 
property of a program, as the number n of its bit-size, seems to be strongly 
related to its halting behavior, and thus, according to the above-mentioned 
mathematical connections, it seems to be intimately linked to the high level, 
mathematical truth encoded in the program. Calude et al. [ 7 ] have recently 
proposed a way to evaluate the difficulty of a hnitely refutable mathemat¬ 
ical problem which is based just on the algorithmic complexity, in a hxed 
language, of the Turing Machine encoding the problem. 

Consider the Riemann hypothesis, for instance |H]. If I am able to show 
that a program of n bits, written to hnd a hnite counterexample, will never 
halt with probability greater than 99.99999999%, then I may safely say that 
Riemann hypothesis is almost certainly true. The singular aspect here is that 
to be able to make such a claim I need only a hnite number of numerical 
checks of the conjecture to reach a probability of 99.99999999%, out of an 
inhnite number of zeros of the zeta function to be checked. Honestly speaking, 
it appears quite surprising. 

The above argument could also be seen as a re-proposition of the Humian 
induction problem [Hj, this time applied to hnitely refutable mathematical 
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statements: if a finitely refutable mathematical statement, encoded in a pro¬ 
gram of n bits of which we are not able to know if it will halt simply inspecting 
its code, holds true for about t ~ steps, then it is dehnitely true with 
a hxed arbitrary high probability. As a matter of fact, all this seems to give 
a strong quantitative support to the inspiring principles of “experimental 
mathematics”, proposed with force by many scholars in the last years ng. 


Acknowledgments 

I wish to thank Prof. Cristian Calude for helpful comments and suggestions. 


References 

[1] Turing, A.M., On Computable Numbers, with an Application to the 
Entscheidungsproblem. Proc. London Math. Soc. Ser., 1936; 42-2: 230- 
265. 

[2] Calude, C. S., B. Pavlov. Coins, quantum measurements, and Turing’s 
barrier. Quantum Information Processing, 2002; 1 (1/2): 107-127. 

[3] Adamyan, V. A., C. S. Calude, B. S. Pavlov. Transcending the limits 
of Turing computability, in T. Hida, K. Saito, S. Si (ed.). Quantum 
Information Complexity. Proeeedings of Meijo Winter Sehool 2003, Sin¬ 
gapore: World Scientihc, 2004, pp. 119-137. 

[4] Chaitin, G.J., Algorithmic Information Theory. Cambridge, UK. Cam¬ 
bridge University Press, 1987. 

[5] Chaitin, G.J., Gomputing the Busy Beaver function. In Open Prob¬ 
lems in Communication and Computation, Ed. by T.M. Gover and 
B. Gopinath, Springer-Verlag, 1987, pp. 108-112. 

[6] Gardner, M., Mathematical Games. Scientific American, pp. 20-34, 
November 1979. A column based on and extensively quoting the 
manuscript “On Random and Hard-to-Describe Number” {IBM Report 
RG 7483) by Gharles H. Bennett. 

[7] Galude, G. S., Elena Galude, M.J. Dinneen. A New Measure of the 
Difficulty of Problems, CDMTCS Research Report, 2006; 277: 20 pp. 

[8] du Sautoy, M. The Music of the Primes. Harper Collins, 2003. 


7 


Hume, D. An enquiry concerning human understanding. Harvard Clas¬ 
sics Volume 37. Collier & Son; 1910. Part I and Part H of Section IV. 


Chaitin, G.J. Randomness in arithmetic and the decline and fall of re- 
dnctionism in pnre mathematics. Chaos, Solitons & Fractals 1995; 5(2): 
143-159. 



